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On examples of Bose-Einstein condensates embedded in two-dimensional optical lattices we show 
that in nonlinear periodic systems modulational instability and inter-band tunneling are intrinsically 
related phenomena. By direct numerical simulations we found that tunneling results in attenuation 
or enhancement of instability. On the other hand, instability results in asymmetric nonlinear tunnel- 
ing. The effect strongly depends on the band gap structure and it is especially significant in the case 
of the resonant tunneling. The symmetry of the coherent structures emerging from the instability 
reflects the symmetry of both the stable and the unstable states between which the tunneling oc- 
curs. Our results provide an evidence of profound effect of the band structure on superfluid-insulator 
transition. 

PACS numbers: 



Bose-Einstein condensate (BEC) embedded in an opti- 
cal lattice represents a remarkable laboratory for study of 
the interplay among various physical phenomena, having 
general physical nature. One of such phenomena- the 
modulational instability - is a characteristic feature of 
nonlinear systems 1]. For BECs (where it is also referred 
to as dynamical instability) it has been described theo- 
retically in Q, IS arid observed experimentally in 
IS . Another phenomenon, which is typical for linear peri- 
odic systems, is the Landau-Zener tunneling. It was also 
discussed in the context of the BEC applications 0, 
and explored in a series of experiments 9], where the 
phenomenon was implemented in one-dimensional (ID) 
optical lattices and was stimulated by the effective lin- 
ear force created by the lattice acceleration. The existing 
theoretical descriptions, based on two- level models, how- 
ever, are not unanimous about how Landau-Zener tun- 
neling occurs, except one point: two-body interactions 
(i.e. the nonlinearity) make the tunneling asymmetric. 
It was suggested in Ref. 8j, that asymmetry stems from 
the fact that the Landau-Zener tunneling in a nonlin- 
ear system is intimately related to the instability. In- 
deed, tunneling couples two band edges with the eigen- 
states that acquire different stability properties in the 
presence of the nonlinearity, one of them being unstable. 
In this case the initial atomic distribution turns out to 
be of crucial importance. Recently, Landau-Zener tun- 
neling and modulational instability have been explored 
within the unique experimental setting 10]. However, 
the data available so far, are not conclusive about the 
reasons for asymmetry of nonlinear Landau-Zener tun- 
neling. Among the other reasons, it is due to the fact 
that to stimulate the tunneling a linear force is neces- 
sary. Such a force gives rise to other phenomena, such 
as, for example, Bloch oscillations g^], which in turn, can 
also imply the asymmetry. 



^From the point of view of understanding the funda- 
mental relation between tunneling and modulational in- 
stability, flexibility of multidimensional lattices in gen- 
eral 11], and 2D lattices, in particular, open new per- 
spectives, offering diversified possibilities of the observing 
tunneling and instabilities, without any use of external 
forces, namely, that of the lattice acceleration. For sys- 
tems of cold noninteracting atoms this has already been 
discussed in Moreover, within the framework of a 

different system, which is a beam propagation through 
an optically induced 2D lattice, nonlinear tunneling has 
been observed experimentally in what follows 

tunneling without external forces is referred to as Bloch- 
band tunneling and is subject of our principal concern. 

Our main goal is to explore intrinsic relation between 
Bloch-band tunneling and modulational instability. We 
show that tunneling can either enhance of attenuate in- 
stability, as well as instability can result in asymmetry 
of tunneling with respect to initial population of stable 
and unstable states. Experimentally such states can be 
created by manipulating with noninteracting atoms and 
subsequent switching in the nonlinearity, what can be 
managed by means of Feshbach resonance. We argue 
that the band structure is the most relevant characteristic 
governing the dynamics and supports resonant exchange 
of the atoms between bands. As an important physical 
application of our results, we notice that instability of 
Bloch waves, with subsequent emergence of coherent lo- 
calized structures, implies the superfluid-insulator transi- 
tion (see e.g. 2], as well as 14] for discussion in the tight- 
binding approximation, and 15] for experimental obser- 
vation). We thus provide an evidence of profound in- 
fluence of the band structure on the superfluid-insulator 
transition and suggest a possibility of its managing by 
simple control of parameters of an optical lattice. 

We consider an effectively 2D BEC, with a positive 
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scattering length, governed by the Gross-Pitaevskii equa- 
tion 

i— = -A^ + V{r)^ + . (1) 

Here time is measured in the units of 2h/ER^ where 
Er = h^k!^ / {2m) is the recoil energy, k = 27r/A is the 
wave vector of the laser beams considered equal in all 
directions, and coordinates r = {x^y) are measured in 
the units of . We restrict our consideration to the 
potential 

V(y) = Vo [cos(2x) + cos{2y) - cos(2x) cos(2?/)] (2) 

where Vq is measured in the units of 2Er. 

When the nonlinearity is weak enough and the lattice 
amplitude is of order of the recoil energy, the underlying 
band structure of V(r) appears to be the most relevant 
characteristic. Then the problem can be described in the 
effective mass approximation 0, . We start with the 
situation where a band structure possesses a full gap in 
all directions (Case III in Fig. The respective linear 
system does not allow tunneling between bands unless 
additional external factors are involved. The nonlinearity 
couples modes in different bands making possible local 
transfer atoms among them |^, . 
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FIG. 1: [Color online] The two lowest bands of the poten- 
tial (0). In the Case I, where Vb = —0.2, the bands overlap: 
Ex - Em ^ -0.045. When Vq = -0.2575 the band struc- 
ture transforms into that shown in the Case II: the gap edges 
acquire practically the same value: Ex - Em ^ 10~^ Fur- 
ther increase of |Vb| leads to opening the gap, as is shown in 
Case III, where Vq = -0.27 and Ex - Em ^ 0.1025. The 
inset in Case I shows notations of the high symmetry points 
in the Brillouin zone. 

When Bloch bands overlap in different directions 
(Case I in Fig. QJ, the linear tunneling between states 
with the same energy is possible. It turns out, however, 
that the most intensive exchange of atoms between the 
bands occurs not in this case, but in the case of a reso- 
nant tunneling^ when matching conditions are satisfied, 
i.e. when four waves satisfy simultaneously the energy 



and momentum conservation laws (see e.g. Tg^). Such 
a lattice can be created by adjusting the laser inten- 
sity as it is shown in Case II of Fig. ^ The states in 
points M (of the first band) and X (of the second band) 
possess not only the same energy Em = Ex [hereafter 
Ea = E{(ia) with £^(q) being the eigenvalue of the oper- 
ator — A + y(x, y); explicit forms of the dependence E{q) 
are shown in Fig.^, but also satisfy the conservation of 
the quasi-momentum. By denoting an arbitrary vector 
of the reciprocal lattice by Q, the last condition can be 
expressed either as qm = qx + qx — qM + Q if one con- 
siders tunneling from initial state X to the state M or as 
qx = qM + qM — qx + Q if one considers tunneling from 
M to X. A process where the majority of particles is ini- 
tially concentrated in the state A from where they tunnel 
into the state B will be referred to as A^B tunneling. 

Resonant wave interactions available in Cases I and 
III, are those conventionally referred to as self-phase and 
cross-phase modulations and governed by the integrals 
Xx,M = 4/y^^x,M^r and x = 4 /^^ ^|^|^c^r, respec- 
tively [hereafter ^m,x are the Bloch states, real for the 
points X and M, and Vq = (0, tt) x (0, tt) is a volume of 
the unit cell of the lattice] . In the two- mode approxima- 
tion one can introduce slowly varying modulations ^x,M 
of the states ^m,x and represent (in the leading order) 



^ = Ax{r,t)^x{r)e 



-iExt 



-^M(r,t)^M(r)e-^^-^ (3) 



Then, following the standard steps (see e.g. 4]), one de- 
duces the Hamiltonian governing the dymnamics slow 
amphtudes {dAa/dt = —iSH/SAa): 



H = 



dr 



MuVAmI 



+Xx l^xl' + Xm I^mI' + 4x l^xl' I^mI'} . (4) 



Here M^^ {a =X,M) is a 2 x 2 tensor of the inverse 
effective mass in a point q^ whose entries are defined 
by M-^^(qc) = d'^E{(ia)/{dqa,idqaj)' The introduced 
dynamics preserves the averaged number of atoms in each 
band, i.e. A^x,m = / |v4.x,Mp<^r are constants. 

If however, the matching conditions are met, then the 
other types of the resonant four- wave interactions, re- 
sulting in change of the average number of atoms in the 
bands, can occur. Now the Hamiltonian governing evo- 
lution of the modulation of the Bloch states reads 



H 



^ y"^ir||MxV^x|V|MMV^M|Vxxl^x 



+Xm I^mI^ + 4x l^xl' I^mI' + xAl^Ajc + xAl^Ajc} .(5) 

The resonant processes occur simultaneously with the 
nonresonant transitions between states in different bands 
but with the same wavevector (as those observed in jl3l]). 
The latter, however, are negligible in our configuration. 
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This was proved numerically by evaluating projections of 
the obtained distributions into the states ^m,x- 

It follows from Q and that stability of initial dis- 
tributions is determined by the signs of 4]. Since 
we consider the condensate with a positive scattering 
length, and M-.i(qx) > and Mt^\cim) < ( j = x,y), 
in the both cases the states and are unsta- 
ble and stable, respectively, since one of the stability 
conditions of a band a populated with the density pa 
reads Z{Z + 2xapl) < with Z = ql/M^, + qf/My. 
Meantime, in the resonant case Q there exists an ad- 
ditional domain of instability defined by the condition 
[Z + pI{3 - Xa)][Z + pI{1 - Xa)] < 0, what results in 
faster development of instability, reported in numerical 
simulations below. Thus, tunneling can either stabilize 
or destabilize the dynamics of the wave function depend- 
ing on the direction of the tunneling that corresponds to 
M^X and X^M processes, respectively. 

To inspect the interplay between tunneling and in- 
stability, we carried out direct numerical simulations of 
Eq. QJ, subject to periodic boundary conditions and ini- 
tial condition in a form o^m^m+ci^x^x [c.f. ©] where the 
factors a determine the distribution of atoms between the 
states. Simultaneous use of smooth initial perturbation 
allowed us to accelerate the development of the instability 
(c.f. time scales in Fig.|2l where no initial perturbation is 
applied and Fig. IHl where initial states were perturbed). 
For quantitative analysis we computed the occupation 
rates rx,M = ^x,m/^, where N = Nx + Nu is the total 
number of atoms. When the two states have nonzero ini- 
tial population, the nonlinearity results in periodic par- 
ticle exchange between these states. When tunneling is 
nonresonant (Cases I and III in Figs. El 01) this exchange 
is relatively weak: it is caused by wave interactions of 
higher orders not accounted by the Hamiltonian . Ex- 
change of atoms between the bands becomes the leading 
order phenomenon when tunneling is resonant (Case II) 
what is described by additional hopping integrals in 0. 

Let us consider X^M process shown in Fig|21 Period 
of oscillations T decreases as amplitude potential |Vb| in- 
creases: from the top to bottom T ^ 560, 362, 116. This 
is explained by growth of the cross-phase modulation: 
(from top to bottom) x ^ 6.4; 6.8; 7.6 x 10"^. When the 
gap is opened (Case III) the system stays for a long time 
in a stable position due to low occupation of the unsta- 
ble state M. Nevertheless, because of periodic increase 
of the population of the state M the instability is de- 
veloped dit t ^ 3100. Possibility of tunneling accelerates 
this process: in the Case I the instability is developed 
at t ~ 2500. The most substantial enhancement of the 
instability is observed when the tunneling becomes reso- 
nant: in the Case II the instability occurs at t ^ 2050. 
The development of the instability results in emergence 
of coherent structures. If tunneling is resonant (Case 
II) atomic distribution displays asymmetric peaks, which 
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FIG. 2: [Color online] (a) Dynamics of the density maximum; 
(b) Time evolution of the occupation rates rM(t) (red lines) 
and rx(t) (blue lines), (c) Snapshots of the atomic density 
distributions after instability is developed. All simulations are 
done with the initial distributions rM(0) = 0.2 and rx(0) = 
0.8. The cases I, II, and III correspond to those in Fig. Q] 



evolve in time. The peaks are elongated in x-direction, 
what seems to contradict to an intuitive assumption that 
the symmetry of the developed coherent structure resem- 
bles the symmetry of the unstable state, from which they 
are evolved 4] . Such an interpretation well corroborates 
with the results shown below in the Cases I and III of 
Figini In the case at hand, however, one has to keep in 
mind that, while the tensor of the inverse effective mass 
is symmetric at the unstable point M, it is not symmetric 
any more at the point X. Specifically, for the Case II in 
Fig.ElMx^ ?^diag(.3, .03). Since M'J^ > M'^, when the 
atoms are at the X-point, dispersion of the wave along the 
x-direction is stronger than the dispersion along ?/-axis. 
Since instability is developed due to particle exchange 
between the two states, this explains elongation of the 
emerging excitations along the x-axis. 

Now we turn to the situation when initially the most 
of particles occupy unstable state M, i.e. to the M^X 
process, illustrated in Fig. [2l In the configuration where 
there is no linear tunneling (Case III) the instability is 
developed oX t ^ 270. Nonresonant tunneling (Case I) 
leads to some attenuation of the instability that starts 
dX t ^ 370. In the both cases the developed coherent 
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FIG. 3: [Color online] The same as in Fig|21 but with ini- 
tial particles distribution between points M and X taken 
as follows: rmiO) = 0.8 and rx(0) = 0.2. To acceler- 
ate modulational instability we impose an initial perturba- 
tion 0.1|^max| sin (0.314x) sin (0.314^) on the numerically ob- 
tained linear Bloch states. 



structures resemble those observed in Ref. 4] for a sepa- 
rable square lattice and reflect the symmetry of the lat- 
tice. Dramatical changes occur in the case of the res- 
onant Bloch-band tunneling (the Case II). Besides sig- 
nificant attenuation of instability, which now occur at 
t ^ 450, one observes change of the symmetry of the de- 
veloped structure. Like in the case of X^M tunneling, 
the emerging pulses are strongly localized along ?/-axis 
and elongated along x-axis, what corroborates with the 
fact that the attenuation of the instability has occurred 
due to tunneling into the stable state X. 

Different symmetries of the emerging structures, that 
can be identified e.g. by means of diffraction imaging, 
provide a possibility for the experimental observation of 
the phenomenon and identification of the type of the pro- 
cess. We also mention that although the time difference 
in development of instability differs in our numerics only 
by 10%^30% for different processes, this is experimen- 
tally observable time. Indeed, for condensed rubidium 
atoms in calculations 50 units of the dimensionless time 
correspond to 2 ms, what means that time difference in 
instability of different regimes is of order of 8^20 ms. 

The results shown in Figs. El [HI reveal a common fea- 



ture, namely that until the instability is developed the 
atoms are mainly distributed between the two intercon- 
nected states and ^x- This is a strong support of 
the validity of a two-mode approximation suggested in 
0, and in this Letter defined by the Hamiltonians 
and 0. However, after the instability is developed, we 
observed transfer of atoms to states other than ^x,m (see 
panels (b) in Figs El 01) • When such transfer takes place 
the two-mode approach is not applicable any more. 

To conclude, we have shown that two-body interac- 
tions in a BEC loaded in a two-dimensional orthogonal 
nonseparable optical lattice result in a strong interplay 
between the phenomena of Bloch-band tunneling and 
modulational instability. As a consequence, tunneling 
appears to be one of the key factors for the superfluid- 
insulator transition, especially in the case of lattices with 
negligible gap, where matching conditions for the four 
wave interactions are provided. The gap width, mean- 
time, is an easily changeable lattice parameters and can 
be controlled by the laser intensity. Specifically we found 
that the atoms, initially loaded in a stable state, due to 
inter-band exchange develop instability, which is signifi- 
cantly enhanced when the matching conditions are sat- 
isfied. On the other hand resonant tunneling attenuates 
the instability when atoms are initially loaded in an un- 
stable state. We also found that the instability makes the 
Bloch-band tunneling strongly asymmetric in the sense 
of dependence of developed patterns on initial atomic 
distribution. 
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